Understanding and manipulating articulated objects, such as doors and drawers, is crucial for robots operating in human environments. We wish to develop a system that can learn to articulate novel objects with no prior interaction, after training on other articulated objects. Previous approaches for articulated object manipulation rely on either modular methods which are brittle or end-to-end methods, which lack generalizability. This paper presents FlowBot++, a deep 3D vision-based robotic system that predicts dense per-point motion and dense articulation parameters of articulated objects to assist in downstream manipulation tasks. FlowBot++ introduces a novel per-point representation of the articulated motion and articulation parameters that are combined to produce a more accurate estimate than either method on their own. Simulated experiments on the PartNet-Mobility dataset validate the performance of our system in articulating a wide range of objects, while real-world experiments on real objects' point clouds and a Sawyer robot demonstrate the generalizability and feasibility of our system in real-world scenarios.

We propose a visionbased system that learns to predict the potential motions of the parts of a variety of articulated objects to guide downstream motion planning of the system to articulate the objects. To predict the object motions, we train a neural network to output a dense vector field representing the point-wise motion direction of the points in the point cloud under articulation. We then deploy an analytical motion planner based on this vector field to achieve a policy that yields maximum articulation. We train a Figure 1: FlowBot3D in action. The system first observes the initial configuration single vision model entirely in simulation across all categories of the object of interest, estimates the per-point articulation of objects, and we demonstrate the capability of our system flow of the point cloud (3DAF), then executes the action based on to generalize to unseen object instances and novel categories in the selected flow vector. Here, the red vectors represent the direction both simulation and the real world using the trained model for of flow of each point (object points appear in blue); the magnitude of all categories, deploying our policy on a Sawyer robot with no the vector corresponds to the relative magnitude of the motion that finetuning. Results show that our system achieves state-of-theart point experiences as the object articulates.

Understanding and manipulating articulated objects such as doors and drawers is a key skill for robots in human environments. However, it is difficult to train systems that generalize to variations of those objects. The sensory signal comes from an Azure Kinect depth camera, and the agent is a Sawyer BLACK robot. A novel per-point representation of the articulation structure of an object is proposed, called 3D Articulation Flow. A newly-developed 3D vision neural network architecture takes as input a static 3D point cloud and predicts the 3D Articulation Flow of the input under articulation motion.